Human behavior recognition is an important basic technology in the fields such as intelligent monitoring, human-computer interaction and robotics. Graph Convolutional Neural Network (GCN) achieve excellent performance in skeleton-based human behavior recognition. The following problems exist in the research of human behavior recognition using GCNs:1) the human skeleton points are represented by coordinates, which lacks detailed information about the movement of the skeleton points; 2) in some videos, the motion amplitude of the human skeleton is too small, so that the representation information of the key skeleton points is not obvious. Aiming at the above problems, firstly, a temporal divergence model of skeleton points was designed to describe the movement states of the skeleton points, which amplified the between-class variances of different human behaviors. In addition, the attention mechanism of temporal divergence features was designed to highlight the key skeleton points and further expand the between-class variances. Finally, a two-stream fusion model was constructed based on the complementarity between the spatial data characteristics of the original skeleton and the temporal divergence characteristics. The proposed algorithm achieved the accuracy of 82.9% and 83.7% under two partitioning strategies of authoritative human behavior dataset NTU-RGB+D respectively, which were 1.3 percentage points and 0.5 percentage points higher than those of Adaptive Graph Convolutional Network (AGCN) respectively. The improvement of the accuracy of the proposed algorithm on the dataset proves the effectiveness of this algorithm.